docs: prune and restructure the guides section - #3483
Conversation
Remove outdated cookbooks and videos, consolidate integration cookbooks into their /integrations reference pages, move benchmarks and the 2023 webinar to /resources/engineering, and sort the remaining guides by topic instead of format. Every removed URL redirects to the closest living equivalent.
|
Check out this pull request on See visual diffs & provide feedback on Jupyter Notebooks. Powered by ReviewNB |
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Move the Ragas evaluation cookbook to /resources/engineering (redirect and inbound links retargeted) and keep the video gallery on the /guides page instead of folding videos into the topic categories.
Resolve redirects.js conflict (keep both the guides cleanup group and the monitors-to-alerts rename from main) and retarget five pre-existing LangGraph cookbook links that the original sweep missed.
|
Too many files changed for review (124 files, 100 file limit). Bypass the limit by tagging |
|
@claude review |
There was a problem hiding this comment.
Beyond the inline findings, I also checked whether example_data_migration-jp being absent from content/guides/cookbook/meta.json's pages array would drop it from the sidebar nav — Fumadocs includes unlisted pages by default (alphabetically, after the explicit list), so this doesn't affect navigation.
Extended reasoning...
This is a large (123-file), design-heavy content restructuring PR (page removals, merges, and ~60 redirects), so it warrants human review regardless of the bugs already flagged inline. I additionally checked the candidate concern that example_data_migration-jp was dropped from content/guides/cookbook/meta.json's pages array and would therefore disappear from the guides sidebar nav. Fumadocs' default meta.json behavior includes any page not explicitly listed (appended alphabetically after the listed pages, since none of this repo's meta.json files use the !name exclusion syntax), so the page remains reachable and listed in the nav — this is not a real issue.
| href="/integrations/model-providers/openai-js" | ||
| arrow | ||
| /> | ||
| <Card | ||
| icon={<Code />} | ||
| title="Notebook" | ||
| href="/guides/cookbook/js_integration_openai" | ||
| arrow | ||
| /> | ||
| </Cards> | ||
|
|
||
| </Tab> |
There was a problem hiding this comment.
🟡 This PR removes the "Notebook" card from two <Cards> blocks in content/docs/observability/get-started.mdx (JS/TS OpenAI section, lines 175-183, and LangChain JS/TS section, ~line 246-253), leaving each with a single <Card>, but both still use num={2}. Since components/docs/cards.tsx only applies full-width col-span-full styling when num={1} (the convention used by the sibling single-card blocks at lines 193 and 264), the lone remaining card in each block now renders at half-width with an empty gap next to it instead of spanning the full row.
Extended reasoning...
The bug is a cosmetic layout regression: two <Cards> blocks in content/docs/observability/get-started.mdx lost their Notebook card in this PR's cookbook-restructuring cleanup, but the num prop on the surrounding <Cards> element was not updated to match the new child count.
Looking at components/docs/cards.tsx (CardsBase, lines 87-95), the component maps num to a grid class: num === 1 gets [&>*]:col-span-full so a lone card spans the full row, num === 3 gets sm:grid-cols-3, and any other value (including the default num === 2) gets no override, meaning Fumadocs' default responsive grid (2 columns at @lg+) is applied. Before this PR, both affected blocks legitimately had two children — the OpenAI JS/TS block had "Docs" + "Notebook" (line ~175-184), and the LangChain JS/TS block had "Docs" + "Notebook" (line ~246-252) — so num={2} was correct at the time.
This PR's diff deletes the Notebook <Card> from both blocks (as part of removing the now-consolidated js_integration_openai and js_integration_langchain cookbooks), leaving a single <Card> inside each <Cards num={2}> wrapper. The file's own convention elsewhere shows the correct fix: the Vercel AI SDK block (line 193) and the Python SDK block (line 264) are both single-card blocks that correctly use <Cards num={1}>, letting the sole card span the full row via col-span-full.
Step-by-step proof:
- Read
components/docs/cards.tsx:87-94— confirmsnum===1→ full-width,num===2(or default) → standard grid, no full-width override. - Read
content/docs/observability/get-started.mdxlines 175-183 — after the diff, this block is<Cards num={2}>containing exactly one<Card>(OpenAI JS docs link), immediately followed by</Cards>. - Read lines 246-253 — same pattern for the LangChain JS/TS block:
<Cards num={2}>with one<Card>after the Notebook card was removed. - Compare to lines 193 and 264 — both are genuinely single-card blocks in the same file, both using
<Cards num={1}>, confirming the file's established convention for this exact situation. - Conclusion: the two affected blocks should also use
num={1}to render consistently with the rest of the page; leavingnum={2}produces a half-width card with an empty gap on desktop viewports.
The impact is purely visual — nothing breaks functionally, no links are wrong, no content is lost. It just leaves an oddly gapped single card on two sections of a docs page. The fix is a one-line change per block: <Cards num={2}> → <Cards num={1}> at both locations.
| href="/docs/sdk/python/example" | ||
| icon={<FileCode />} | ||
| /> | ||
| <Card | ||
| title="Video introduction" | ||
| href="/guides/videos/introducing-python-decorator" | ||
| icon={<VideoIcon />} | ||
| /> | ||
| </Cards> |
There was a problem hiding this comment.
🟡 In content/blog/2024-04-python-decorator.mdx, the PR removes the 'Video introduction' Card (icon={}, linking to the deleted /guides/videos/introducing-python-decorator page) but leaves the now-unused Video as VideoIcon import from lucide-react on line 226. Trim it to import { FileCode, BookOpen } from "lucide-react";.
Extended reasoning...
The PR deletes the Video introduction <Card> at the end of this blog post, which was the only place using <VideoIcon /> (it linked to /guides/videos/introducing-python-decorator, a page also removed in this PR). However, the import statement on line 226 was left unchanged: import { FileCode, BookOpen, Video as VideoIcon } from "lucide-react";.
A grep for VideoIcon across the file confirms it now appears exactly once — in that import statement — with zero remaining JSX usages. Note that the '<Video src=... />' tags elsewhere in the file (e.g. lines 44 and 56) are unrelated: those reference the project's custom MDX video-embed component, not the lucide-react icon that gets aliased to VideoIcon here. So there is no other consumer that would make this import still necessary.
FileCode and BookOpen are both still referenced by the two remaining <Card> elements, so only VideoIcon needs to be dropped. The fix is a one-line change:
- import { FileCode, BookOpen, Video as VideoIcon } from "lucide-react";
+ import { FileCode, BookOpen } from "lucide-react";Step-by-step proof:
- Before this PR, the file had three Cards: Docs, Example notebook, and Video introduction (the last using
icon={<VideoIcon />}). - This PR deletes the Video introduction Card block entirely (visible in the diff), leaving only the Docs and Example notebook Cards.
- The import line itself was not touched by the diff, so
Video as VideoIconis still imported. - Since no JSX in the file references
VideoIconanymore, it is dead code — an artifact of the Card removal rather than an intentional import.
This is a pure MDX cleanup: an unused import in an MDX frontmatter-adjacent JS import doesn't break the build, doesn't affect rendering, and doesn't fail any CI check (H1 check, Prettier, link-check are all unaffected). It's a straightforward nit a maintainer would want cleaned up during review, but it does not block merging.
| **The answer is, you can use RAGAS Evaluations with Langfuse.** We have written a tutorial that shows how to [run RAGAS evaluations](/resources/engineering/evaluation-of-rag-with-ragas) on your historical LLM data. Additionally, you can use the RAGAS evaluation prompts and use them with [Langfuse's automated eval service](/docs/evaluation/evaluation-methods/llm-as-a-judge) to run RAGAS evaluations on any new LLM trace. | ||
|
|
||
| ## 16. Can I use Langchain, Langserve and Langgraph with Langfuse Observability and Tracing? | ||
|
|
||
| **Yes, you can use [Langchain](/integrations/frameworks/langchain), [Langserve](/integrations/frameworks/langserve) and [Langgraph](/integrations/frameworks/langchain) with Langfuse to gain observability, tracing and evals.** Langfuse integrates with both Langchain Python and Langchain JS. Langfuse has maintained a first class integration with Langchain since day one. | ||
| **Yes, you can use [Langchain](/integrations/frameworks/langchain) and [Langgraph](/integrations/frameworks/langgraph) with Langfuse to gain observability, tracing and evals.** Langfuse integrates with both Langchain Python and Langchain JS. Langfuse has maintained a first class integration with Langchain since day one. |
There was a problem hiding this comment.
🟡 FAQ heading 16 still says "Can I use Langchain, Langserve and Langgraph..." but this PR rewrote the answer to mention only Langchain and Langgraph, removing the Langserve link entirely (since /integrations/frameworks/langserve was deleted and now redirects to LangChain). A reader specifically looking for Langserve support gets no answer. Suggest dropping "Langserve" from the heading, or adding a sentence noting Langserve support was folded into the LangChain integration.
Extended reasoning...
The FAQ heading at content/faq/all/fifteen-questions-langfuse-answered.mdx:106 reads: "## 16. Can I use Langchain, Langserve and Langgraph with Langfuse Observability and Tracing?" This heading was not touched by this PR's diff, but the answer directly below it (line 108) was rewritten as part of this PR's broader effort to remove the deprecated LangServe integration (per the PR description: "LangServe is deprecated since Nov 2024… Redirects to LangChain integration").\n\nBefore this PR, the answer read: "Yes, you can use Langchain, Langserve and Langgraph with Langfuse..." After this PR, it reads: "Yes, you can use Langchain and Langgraph with Langfuse to gain observability, tracing and evals." The Langserve mention and its link were dropped entirely, since the target page /integrations/frameworks/langserve no longer exists (it now redirects to the LangChain integration page).\n\nThis creates a heading/body mismatch: the question explicitly asks about Langserve, but the answer no longer addresses it at all — not even to note that Langserve is deprecated or that its functionality moved to the LangChain integration. A reader who lands on this FAQ entry specifically because they're wondering about Langserve support gets no information relevant to their question.\n\nStep-by-step to reproduce the inconsistency:\n1. Open content/faq/all/fifteen-questions-langfuse-answered.mdx after this PR merges.\n2. Read heading 16: "Can I use Langchain, Langserve and Langgraph with Langfuse Observability and Tracing?"\n3. Read the answer immediately below: it only confirms Langchain and Langgraph support — no mention of Langserve, deprecation, or a pointer to the LangChain integration as the replacement.\n4. A reader who searches this FAQ for "Langserve" (e.g., via page search or an AI chat answer sourced from this FAQ) will be told the question covers Langserve, but find no actual answer about it.\n\nThis wasn't caught by the existing content because nothing else in the file references this heading, and no automated check (link-check, h1-check) would flag a plain-text heading/body mismatch — it's purely an editorial oversight from editing the answer without revisiting the question.\n\nFix: either (a) drop "Langserve" from the heading so it reads "Can I use Langchain and Langgraph with Langfuse Observability and Tracing?", matching the answer, or (b) keep the heading and add a short clause to the answer noting Langserve support was folded into/superseded by the LangChain integration, consistent with how the PR handled the Langserve removal elsewhere on the site (e.g. redirecting /integrations/frameworks/langserve to the LangChain page).\n\nThis is a purely cosmetic content-consistency issue — no link is broken, no anchor breaks, and the page still builds and renders correctly. All three independent verifiers agreed this is real but rated it "nit" severity for the same reason.
| - Use [Langfuse Prompt Management](https://langfuse.com/docs/prompts/get-started) and link prompts to traces | ||
| - Add [score](https://langfuse.com/docs/scores/custom) to traces | ||
|
|
||
| Visit the [OpenAI SDK cookbook](https://langfuse.com/guides/cookbook/integration_openai_sdk) to see more examples on passing additional parameters. | ||
| Visit the [OpenAI SDK integration docs](/integrations/model-providers/openai-py#custom-trace-properties) to see more examples on passing additional parameters. |
There was a problem hiding this comment.
🟡 This PR retargets the dead /guides/cookbook/integration_openai_sdk link in xai-grok.mdx, but xai-grok.mdx is generated from cookbook/integration_x_ai_grok.ipynb (per cookbook/_routes.json), and the notebook still has the old broken link. Since scripts/move_docs.py unconditionally overwrites the .mdx from the notebook on every run of update_cookbook_docs.sh, the next routine regeneration will silently revert this fix. Per the repo's own workflow rule (edit notebook sources in cookbook/, not generated files), the link update should be moved into cookbook/integration_x_ai_grok.ipynb.
Extended reasoning...
This PR edits content/integrations/model-providers/xai-grok.mdx directly, changing https://langfuse.com/guides/cookbook/integration_openai_sdk to /integrations/model-providers/openai-py#custom-trace-properties. However, this .mdx file is not a hand-authored source — it is generated output. cookbook/_routes.json maps integration_x_ai_grok.ipynb to docsPath: "integrations/model-providers/xai-grok", and the file carries the source: ⚠️ Jupyter Notebook frontmatter marker confirming this.
scripts/move_docs.py builds the destination path from docsPath and unconditionally overwrites it via open(full_destination_path, "w") — there is no diff-and-skip or notebook-is-newer check, and no special-casing based on whether the destination lives under content/guides/cookbook/ versus elsewhere in content/integrations/. scripts/update_cookbook_docs.sh invokes this conversion on every run, which is a routine, unrelated maintenance task documented in this repo's own AGENTS.md/CLAUDE.md.
The notebook source, cookbook/integration_x_ai_grok.ipynb, still contains the old link and was not touched by this PR (confirmed absent from the diff). So the code path that will actually run next is: someone runs bash scripts/update_cookbook_docs.sh for an unrelated cookbook change → move_docs.py regenerates every mapped .mdx from its notebook, including xai-grok.mdx → the freshly-generated file overwrites the hand-edited fix with the stale notebook content → the link reverts to /guides/cookbook/integration_openai_sdk, a page this very PR deletes.
Nothing in the current CI or build catches this today: the link-check job only validates the live .mdx content at the time of the PR, and since the hand-edit currently holds the correct target, pnpm link-check and the build pass cleanly on this PR. The regression is deferred to the next regeneration cycle, not immediate. It is also softened somewhat by lib/redirects.js, which contains a redirect from /guides/cookbook/integration_openai_sdk to /integrations/model-providers/openai-py — so on regen the link would degrade to a redirect (losing the #custom-trace-properties anchor) rather than hard-404. Still, this is a real, silent regression of the PR's own goal and a direct violation of the repository's explicit workflow rule: "For cookbook changes, edit notebook sources in cookbook/... Do not hand-edit generated files in content/guides/cookbook/" (and by extension, files elsewhere in content/ that are also cookbook-derived per _routes.json).
Proof by walkthrough:
- Today:
content/integrations/model-providers/xai-grok.mdxline ~87 contains the new, correct link (per this PR's diff).cookbook/integration_x_ai_grok.ipynbline ~150 still contains the old link (unchanged, not in this PR's diff). - A future contributor makes an unrelated fix to a different cookbook and runs
bash scripts/update_cookbook_docs.shper the documented workflow. - That script invokes
scripts/move_docs.py, which iterates all entries incookbook/_routes.json, including the xai-grok entry, and regenerates markdown from every notebook, writing each withopen(dest, "w")regardless of whether the destination content actually changed. content/integrations/model-providers/xai-grok.mdxis overwritten with content derived from the still-stale notebook, silently reintroducing the old, now-redirected link and discarding this PR's fix.- No test or CI check catches this, because the regenerated
.mdxstill links to something valid (a redirect target), solink-checkcontinues to pass — the regression is invisible short of manually diffing the regenerated file against the notebook.
Fix: Apply the same link retarget inside cookbook/integration_x_ai_grok.ipynb (the cell containing https://langfuse.com/guides/cookbook/integration_openai_sdk), then regenerate xai-grok.mdx via bash scripts/update_cookbook_docs.sh so the hand-edit and the generated output are consistent with the notebook as the source of truth.
| description: Use of Langfuse decorator to trace calls made to openai assistants | ||
| category: Integrations | ||
| logo: /images/integrations/openai_icon.svg | ||
| logoAppearance: dark | ||
| --- | ||
|
|
||
| # Cookbook: Observability for OpenAI Assistants API with Langfuse | ||
|
|
||
| > **Note:** OpenAI has deprecated the Assistants API and will shut it down on August 26, 2026. For new projects, use the [Responses API](https://platform.openai.com/docs/guides/migrate-to-responses) instead. | ||
|
|
||
| This cookbook demonstrates how to use the Langfuse [`observe` decorator](https://langfuse.com/docs/observability/sdk/instrumentation#observe-wrapper) to trace calls made to the [OpenAI Assistants API](https://platform.openai.com/docs/assistants/overview). It covers creating an assistant, running it on a thread, and observing the execution with [Langfuse tracing](https://langfuse.com/docs/tracing). | ||
|
|
||
| Note: The native [OpenAI SDK wrapper](https://langfuse.com/integrations/model-providers/openai-py) does not support tracing of the OpenAI assistants API, you need to instrument it via the decorator as shown in this notebook. |
There was a problem hiding this comment.
🟡 This page uses logo: /images/integrations/openai_icon.svg but the PR drops the logoAppearance: dark frontmatter field that was previously set (and still is on sibling pages openai-py.mdx/openai-js.mdx). Without it, the black OpenAI logo won't invert in dark mode on the integrations catalog. Since this .md is generated from cookbook/integration_openai_assistants.ipynb, the fix should add logoAppearance: dark to the notebook frontmatter (which also lacks it), not the generated .md.
Extended reasoning...
What the bug is: content/integrations/model-providers/openai-assistants-api.md sets logo: /images/integrations/openai_icon.svg in its frontmatter but this PR's diff removes the accompanying logoAppearance: dark line (visible as a removed line right next to the added deprecation-note change). The two other pages that use the exact same openai_icon.svg — openai-py.mdx and openai-js.mdx — both keep logoAppearance: dark in their frontmatter.
Why it matters: components/integrations/IntegrationIndex.tsx reads the logoAppearance field and, when it equals "dark", applies Tailwind classes dark:brightness-0 dark:invert to the logo <img>. This is how black logos (like the OpenAI mark) get inverted to white/visible when the site is in dark mode. source.config.ts declares logoAppearance as an enum (dark/light/multicolor) precisely to drive this behavior consistently across integration cards.
Concrete proof / step-by-step:
- Before this PR,
content/guides/cookbook/integration_openai_assistants.md(the old dual-published guides copy) and the docs copy both carriedlogoAppearance: dark— added in a prior commit (748bd40, Jul 28). - This PR (5298441) regenerates
content/integrations/model-providers/openai-assistants-api.mdfromcookbook/integration_openai_assistants.ipynb, whose notebook frontmatter never includedlogoAppearance. Regeneration silently drops the field that had been hand-maintained on the generated file. - The PR's stated goal for this file was only to add a deprecation note (OpenAI Assistants API shuts down Aug 26, 2026) pointing to the Responses API migration guide — the
logoAppearanceremoval is an unrelated, unintentional side effect of regeneration. - Result: on the integrations catalog/sidebar, in dark mode, the OpenAI icon on this specific card renders as plain black (no invert), while every other OpenAI-branded card (openai-py, openai-js, openai-agents, codex) correctly inverts to white via
dark:brightness-0 dark:invert. This is a visible inconsistency limited to this one page.
Why existing code doesn't catch it: there's no validation tying logo to a required logoAppearance; the field is optional in the schema, so a missing value just silently falls back to no dark-mode adjustment rather than erroring.
How to fix: per this repo's cookbook workflow (docs generated from notebooks must not be hand-edited), add logoAppearance: dark to the frontmatter cell of cookbook/integration_openai_assistants.ipynb and regenerate via bash scripts/update_cookbook_docs.sh, rather than patching the generated .md directly.
Severity: this is purely a cosmetic dark-mode contrast regression — nothing breaks functionally, and it's isolated to one integration card's logo rendering. Recommending nit.
Resolve conflicts from the guides prune (#3483): drop deleted langchain/uptrain cookbooks, keep v4 API notebook updates, and preserve main's category/cross-language cookbook metadata. Co-authored-by: Cursor <cursoragent@cursor.com>
Restructures the guides section following the docs cleanup decisions from the 2026-08-05 marketing sync: remove outdated content, keep integration material in
/integrations(one page per integration, no parallel cookbook entries), move benchmarks, webinars, and evaluation-library write-ups to/resources/engineering, and sort the remaining cookbooks by topic. Review feedback from the first pass is incorporated (Ragas cookbook moved to resources; the video gallery stays on/guides).Net result:
/guidesgoes from 50 cookbooks + 13 videos in 8 fuzzy categories to 18 cookbooks in exactly three topic categories (Evaluation, Examples, Prompt Management) plus a gallery of the 4 current videos. The integrations catalog gains 5 real pages, engineering resources gains 4. Every removed URL redirects to the closest living equivalent, all internal links are retargeted, and all pages and redirects were verified against a local dev server.Removed from the site (outdated or redundant)
guides/cookbook/integration_langserve+integrations/frameworks/langserveguides/cookbook/integration_llama-index-callbackguides/cookbook/integration_llama-index_instrumentationguides/cookbook/integration_llama-index_milvus-lite/integrations/other/milvusguides/cookbook/integration_llama_index_posthog_mistralguides/cookbook/evaluation_with_uptrainguides/cookbook/evaluation_with_langchainguides/cookbook/example_decorator_openai_langchainguides/cookbook/integration_langchain+js_integration_langchainguides/cookbook/integration_openai_sdkguides/cookbook/integration_openai_structured_output#structured-outputguides/cookbook/integration_azure_openai_langchainguides/videos/introducing-langfuse-2.0,introducing-datasets-v2,introducing-python-decorator,posthog-integration,llm-playground,llm-as-a-judge-eval-on-dataset-experimentsguides/videos/run-langfuse-locally,external-evaluation-pipelinesMerged into the integration reference page, then removed
integration_litellm_proxy+js_integration_litellm_proxyintegration_databricksjs_integration_openaiMoved to a better home (content unchanged, URL redirects)
otel_integration_openllmetry,otel_integration_openlit,otel_integration_mlflow,otel_integration_arize/integrations/other/(OpenLLMetry, OpenLIT, MLflow, OpenInference)integration_langgraph/integrations/frameworks/langgraphevaluation_of_rag_with_ragas/resources/engineering/evaluation-of-rag-with-ragaslangfuse_sdk_performance_test,prompt_management_performance_benchmark/resources/engineering/guides/videos/webinar-observability-llm-systems/resources/engineering/No longer dual-published on /guides
These lived at two URLs with identical content; they now exist only as their
/integrationspage (old URL redirects): LlamaIndex, LlamaIndex Workflows, Amazon Bedrock, Anthropic (Python), Anthropic (JS), and OpenAI Assistants API. The Assistants page additionally gets a deprecation note (OpenAI shuts the API down on August 26, 2026) pointing to the Responses API migration guide.Kept on /guides, edited in place
guides/index.mdxexample_evaluating_openai_agents,example_langgraph_agentsexample_data_migration+example_data_migration-jpexample_intent_classification_pipeline,example_llm_security_monitoring,js_prompt_management_langchainLink fixes and plumbing
lib/redirects.js: 41 new redirects, 20 existing entries retargeted so nothing points at a deleted page.cookbook/_routes.jsonreduced to 84 entries; five navmeta.jsonfiles updated.Open follow-ups (deliberately not in this PR):
guides/cookbook/prompt_management_openai_functionsstill uses the legacy OpenAI functions API; refresh to tool calling or delete.example_multi_modal_traces) will come as a separate PR.Note
Medium Risk
Large-scale URL and link changes risk broken inbound links or missed redirect targets, but scope is documentation-only with explicit redirects and no runtime code paths.
Overview
This PR restructures the guides section so integration how-tos live on
/integrations(one page per integration) and benchmarks, webinars, and library-specific eval write-ups move to/resources/engineering./guidesdrops from ~50 cookbooks to 18, grouped into Evaluation, Examples, and Prompt Management, with the video gallery trimmed to four current videos.Removed or relocated content includes deprecated LangServe cookbooks, duplicate LlamaIndex notebooks, unmaintained UpTrain/LangChain eval cookbooks, legacy OpenAI/LangChain cookbook duplicates (now pointed at integration docs), and old launch videos (changelog/blog embeds remain). LangGraph, RAGAS, SDK/prompt benchmarks, and the observability webinar get new canonical URLs under integrations or engineering resources.
Plumbing:
lib/redirects.jsadds many redirects so old cookbook URLs resolve; ~25 other MDX files only change hrefs (academy, blog, changelog, docs, FAQ). A few remaining cookbooks get category metadata and cross-links (e.g. data migration EN/JP). Integration reference pages absorb merged material (LiteLLM proxy tracing, Databricks tabs, OpenAI JS trace grouping) where cookbooks were deleted.Reviewed by Cursor Bugbot for commit 548efe0. Bugbot is set up for automated code reviews on this repo. Configure here.